Platform Explorer / Nuxeo Platform 2021.51

Bundle org.nuxeo.elasticsearch.audit

In bundle group org.nuxeo.elasticsearch

Documentation

  • README.md

    nuxeo-elasticsearch-audit

    About

    This project provides a backend based on Elasticsearch for Nuxeo for Audit Service.

    The idea is to use Lucene / Elastocsearch as storage backend for the Audit trail entries.

    Usage of Elasticsearch as a replacement for the JPA based default backend allows to easily make the Audit service scale :

    • when there are a lot of entries
      • lot of access
      • custom logging
    • when there are a lot of queries
      • reporting
      • usage of sync systems like Nuxeo Drive
    • when custom attributes on Audit entries are used

    How it works

    An Elasticsearch based AuditBackend is contributed at startup to replace the default JPA based one and it use dedicated audit Elasticsearch index to handle storage and queries.

    The queries and PageProviders are based on Elasticsearch native DSL : no automatic conversion is done between EJBQL and Elasticsearch DSL.

    The orginal Audit service uses a JPA sequence to assign each audit entry a unique id.

    In the Elasticsearch implementation, an alternate sequence genaration system is used : nuxeo-elasticsearch-seqgen.

    Data Migration

    When nuxeo-elasticsearch-audit is deployed it will automatically replace the default JPA implementation.

    However, if you have previous data inside JPA you have to migrate it.

    The migration is done in a Worker that will simply go through all existing JPA Audit entries to store them inside the Elasticsearch index.

    The migration is done using a configurable batch size (default is 1000 entries).

    An Automation Operation is provided to trigger the migration Work.

    Sample call using curl :

    curl -H 'Content-Type:application/json' -X POST -d '{"params":{"batchSize":5000}}' -u Administrator:Administrator http://127.0.0.1:8080/nuxeo/api/v1/automation/Audit.StartMigration
    

    NB : Migration of 16 Millions entries on a C3.XLarge AWS instance with default PGSQL setup and default embedded Elasticsearch takes about 3h (1500 entries/s).

    Building

    To build and run the tests, simply start the Maven build:

    mvn clean install
    
  • Parent Documentation: README.md

    nuxeo-elasticsearch

    About

    This project aims at providing Nuxeo bundles to integrate Elasticsearch with the Nuxeo Platform.

    The idea is to index/query Nuxeo Documents to/from Elasticsearch.

    Building

    To build and run the tests, simply start the Maven build:

    mvn clean install
    

    Running tests

    Using the embedded Elasticsearch

    With the default RestClient:

    mvn -nsu test
    # -Dnuxeo.test.elasticsearch.client=RestClient is set by default
    

    Or with the TransportClient

    mvn -nsu test -Dnuxeo.test.elasticsearch.client=TransportClient
    

    Using an external Elasticsearch

    1. Start an Elasticsearch using docker:
    docker run -p 9400:9200 -p 9600:9300 docker.elastic.co/elasticsearch/elasticsearch-oss:6.5.3
    

    2.a Run the test with RestClient:

    mvn -nsu test -Dnuxeo.test.elasticsearch.addressList=http://localhost:9400
    

    2.b Or run the test with TransportClient:

    mvn -nsu test -Dnuxeo.test.elasticsearch.client=TransportClient -Dnuxeo.test.elasticsearch.clusterName=docker-cluster -Dnuxeo.test.elasticsearch.addressList=localhost:9600
    

    Links

    • Administration: http://doc.nuxeo.com/x/UBY5AQ
    • Configuring mapping: http://doc.nuxeo.com/x/WxI5AQ
    • Overview: http://doc.nuxeo.com/x/iYElAQ
    • Open jira tikets

Resolution Order

[202, 824]
The resolution order represents the order in which components have been resolved by the Nuxeo Runtime framework. This range represents the minimal and maximal orders for this bundle's components.
You can influence this order by adding "require" tags in the component declaration, to make sure it is resolved after another component. It will also impact the order in which contributions are registered on their target extension point (see "Registration Order" on contributions).

Components

Maven Artifact

Filenuxeo-elasticsearch-audit-2021.51.1.jar
Group Idorg.nuxeo.elasticsearch
Artifact Idnuxeo-elasticsearch-audit
Version2021.51.1

Manifest

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: root
Build-Jdk: 11.0.22
Bundle-ManifestVersion: 1
Bundle-Version: 1.0.0
Bundle-Name: Audit ElasticSearch backend
Bundle-SymbolicName: org.nuxeo.elasticsearch.audit;singleton:=true
Nuxeo-Component: OSGI-INF/elasticsearch-audit-contrib.xml,OSGI-INF/es-
 audit-pageprovider-contrib.xml,OSGI-INF/es-audit-search-contrib.xml

Exports

Charts

    Raw Data: Json Contribution Stats

    Contributions by Code Type

    Loading data

    Contributions by Target Extension Point

    Loading data

    Contributions by Studio Source

    Loading data